Lecture 5 : Introduction to ( Robertson / Spärck Jones ) Probabilistic Retrieval

In this lecture, we will introduce our second paradigm for document retrieval: probabilistic retrieval. We will focus on Roberston and Spärck Jones’ 1976 version, presented in the paper Relevance Weighting of Search Terms. This was an influential paper that was published when the Vector Space Model was first being developed — it is important to keep in mind the differences and similarities between these two models and the motivations for each. Recall that the Vector Space Model was originally a representation model. The retrieval scheme of the Vector Space Model was empirically-driven and chosen in a fairly atheoretical manner. In contrast, probabilistic retrieval is more principled and theoretically-driven. On the other hand, many statistical estimations and empirical substitutions will drive the derivation of this paradigm.